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ABSTRACT 

Two  new  applications  of  established  techniques  for  mea- 
suring an  individual's  level  of  stress  (workload)  in  tracking 
tasks  are  presented.   An  indirect  technique  of  measuring 
"reserve  capacity"  is  utilized  in  a  two-axis  cross-coupled 
compensatory  tracking  task.   A  direct  psychophysiological 
measurement  is  made  by  recording  time  histories  of  operator 
pupil  diameter. 

Results  obtained  indicate  that  each  method  yields  a  good 
index  of  workload,  although  considerable  variance  in  the  data 
is  observed.   The  level  of  instability  in  the  second  axis  of 
the  cross-adaptive  method  is  shown  to  be  related  to  the  level 
of  workload  in  the  primary  axis.   Increased  pupil  diameter  is 
shown  to  be  similarly  related,  to  operator  workload.   The 
simultaneous  application  of  both  techniques  is  determined  to 
be  inappropriate. 
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I.   INTRODUCTION 

As  the  performance,  complexity  and  cost  of  modern  aircraft 
weapons  systems  continue  to  accelerate,  the  physical  and 
mental  demands  imposed  upon  their  pilots  become  unmanageable. 
To  operate  within  the  narrowing  margins  of  safety  and  effec- 
tiveness, pilots  can  no  longer  tolerate  large  attentional 
demands . 

The  development  of  boosted  hydraulic  control  systems  has 
reduced  the  physical  loads  on  the  pilot  to  a  minimum,  but  the 
aggregate  of  mental  loads,  although  reduced  significantly  in 
recent  years,  has  hardly  reached  its  nadir.   The  level  of 
research  activity  in  this  area  of  workload  reduction  continues 
to  remain  high  and  the  development  of  related  equipment  has 
progressed  from  simple  autopilots  to  complex  inertial  guidance 
and  navigation  systems,  from  gyro  horizons  to  situation  indi- 
cators and  flight  directors,  from  the  well-known  "joy  stick" 
to  the  new  side-mounted  hand  controller.   These  devices  have 
all  been  designed  to  increase  the  pilot's  performance  in  the 
modern  arena  of  the  air  by  reducing  his  attentional  demands; 
i.e.,  his  mental  workload. 

The  term  workload  refers  here  to  a  level  of  stress  or  the 
amount  of  effort  expended  to  achieve  a  certain  level  of 
performance.   Workload  and  performance  are  thus  inter-connected 
but  distinct.   Kelley  [Ref.  1]  has  shown  that  performance 
measurements  are  not  in  themselves  valid  workload  indicators 
and  that  under  certain  circumstances  increased  performance 


can  be  achieved  at  higher  levels  of  performance-measured  work- 
load, and  vice  versa.  Reference  2  similarly  demonstrates  this 
fact  during  an  evaluation  of  helicopter  autopilots. 

This  important  distinction  between  workload  and  performance 
can  be  exemplified  by  the  case  of  two  jet  aircraft  performing 
a  night  carrier  landing.   One  of  the  aircraft  is  equipped  with 
an  approach  power  compensator  (an  automatic  throttle),  and  the 
other  is  not.   Now  consider  the  pilots-.   Both  must  achieve  the 
same  high  level  of  performance  to  bring  the  aircraft  aboard 
but  their  levels  of  workload  are  considerably  different. 

Thus,  in  order  to  properly  evaluate  an  aircraft  system  or 
the  effect  of  changes  within  that  system,  both  operator 
performance  and  workload  measurements  must  be  taken. 

Methods  of  measuring  performance  are  now  quite  standard 
and  commonly  use  some  function  of  "system  error"  as  the 
criterion.   Methods  of  determining  workload,  however,  have 
yet  to  be  completely  developed.   There  exists  no  "meter"  which 
when  plugged  into  the  complex  man-machine  system  gives  direct 
readings  of  workload.   The  various  techniques  which  have  been 
employed  in  past  research  can  be  grouped  into  three 
categories : 

1.  subjective  evaluations  (pilot  ratings) 

2.  measurement  of  "reserve  capacity" 

3.  measurement  of  psychophysiological  responses 

Singly  and  in  combination,  these  techniques  have  been  shown 
to  be  relatively  reliable  indicators  of  workload. 
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Pilot  ratings,  when  properly  used,  have  been  regarded  as 
the  most  effective  of  these  techniques  and  have  been  used  to 
correlate  or  validate  the  other  workload  measures  [Refs.  3,4]. 
By  definition,  however,  these  ratings  are  operator-variant  and 
require  a  certain  level  of  experience  upon  which  to  base  the 
evaluation.   In  view  of  these  requirements  and  in  light  of 
previous  validation  of  the  methods  employed  in  this  research 
by  pilot  rating  [Refs.  4,5],  this  technique  was  not  investigated 

Thus,  in  an  attempt  to  provide  a  simple,  operator-invariant 
method  of  workload  measurement,  this  thesis  has  made  use  of 
techniques  (2)  and  (3),  above.  The  application  of  a  cross- 
adaptive  unstable  tracking  task  was  used  to  measure  reserve 
capacity  and  the  recording  of  changes  in  pupil  diameter  was 
used  as  the  psychophysiological  measure.  Both  methods  were 
examined  while  subjects  performed  a  compensatory  tracking 
task. 
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II.   THEORY 

A.   RESERVE  CAPACITY 

In  the  field  of  manual  control  the  term  "reserve  capacity" 
(or  "excess  control  capacity")  is  used  to  indicate  the  differ- 
ence between  an  operator's  maximum  control  capacity  and  that 
amount  taken  up  by  the  particular  task  he  is  performing.   The 
normalized  ratio  of  this  absorbed  amount  to  the  total  amount 
of  control  is  the  attentional  demand  of  the  task,  or  simply 
the  level  of  workload  the  task  places  on  the  operator.   Thus 
by  measuring  this  reserve  capacity  one  is  led  to  the  "level 
of  workload" : 

T    ,   „....,   ,   attentional  demand    ,    reserve  caoacity 

Level  of  Workload  =  — t—z — i rr =  1  -   ,  , — ^ ■ — r— — *- 

total  capacity  total  capacity 

(1) 
Note  that  the  maximum  level  of  workload,  as  defined  here,  is 
unity. 

1 .   Operator  Loading  Tasks 

The  most  common  method  of  determining  the  reserve 
capacity  is  to  load  an  operator  with  an  additional,  secondary 
task.   This  task  may  take  several  forms;  a  separate  tracking 
task,  a  mental  task  (e.g.  solving  mathematical  problems),  or 
a  responsive  task  (e.g.  responding  to  light  signals  by  pushing 
the  correct  button). 

The  reserve  capacity  of  the  operator  can  be  measured 
in  several  ways .   In  the  method  most  frequently  used,  the 
operator's  performance  on  the  primary  (original)  task  is 
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determined,  from  this  a  criterion  is  established,  and  the 
operator  is  told  to  maintain  this  level  of  performance 
throughout  the  task.   The  level  of  difficulty  of  the  secondary 
(loading)  task  is  increased  until  the  operator  reaches  the 
point  where  he  can  no  longer  meet  the  primary  task  performance 
criterion.   This  level  of  secondary  task  difficulty  then 
becomes  the  measure  of  the  operator's  reserve  capacity. 

Knowles  [Ref.  6]  provides  an  excellent  summary  of 
this  loading  rationale  and  of  early  work  in  the  area. 
2 .   Cross-Adaptive  Systems 

A  problem  frequently  encountered  in  using  these 
secondary  loading  techniques  is  the  difficulty  of  constraining 
the  primary  task  performance.   Although  instructed  otherwise, 
an  operator  may  divert  his  attention  from  the  primary  task  and 
focus  on  the  secondary,  particularly  when  the  secondary  level 
of  difficulty  is  high.   Thus  his  primary  task  performance 
deteriorates  and  the  resulting  higher  level  of  difficulty  he 
can  tolerate  in  the  secondary  task  is  no  longer  a  valid 
measure  of  his  reserve  capacity. 

The  so-called  cross-adaptive  technique  overcomes  this 
problem  by  automatically  adjusting  the  level  of  difficulty  on 
the  secondary  task  on  the  basis  of  primary  task  performance. 
Although  the  operator  may  become  distracted  by  a  very  diffi- 
cult secondary  task  causing  a  lower  level  of  performance  in 
the  primary,  the  adaptive  system  will  sense  the  poorer  per- 
formance and  reduce  the  level  of  difficulty.   The  distraction 
is  thus  reduced  and  the  primary  task  performance  returns  to 
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the  desired  level.  In  this  way  the  operator  is  constrained 
to  maintain  the  prescribed  level  of  primary  performance  and 
an  accurate  measure  of  his  reserve  capacity  can  then  be  made. 

A  schematic  of  this  type  system  is  shown  in  Fig.  (1). 

Kelley  [Ref.  7]  points  out  that  there  are  three 
elements  of  such  an  adaptive  system.   They  are:   (1)  a  method 
of  measuring  performance,  (2)  an  adjustable  parameter  which 
changes  the  level  of  difficulty  (called  the  adaptive  variable) 
and  (3)  the  adaptive  logic  which  automatically  changes  the 
adaptive  variable  as  a  function  of  the  performance  measurement. 

The  specific  application  of  these  elements  to  this 
thesis  is  described  in  Section  III. 
3.   Critical  Instability  Task 

In  a  report  which  is  generally  regarded  as  a  classic 
in  the  field  of  control  theory,  McRuer  et  al.  [Ref.  8]  pre- 
sented a  complex  mathematical  formulation  of  a  human  operator 
in  a  compensatory  tracking  task.   This  quasi-linear  model 
consists  of  a  linear  describing  function  and  the  remaining 
non-linearities  which  are  grouped  and  referred  to  as  "remnant." 
This  remnant  is  thought  to  be  caused  by  non-linear  or  unsteady 
operator  behavior  and  by  the  superposition  of  noise  on  the 
operator's  linear  output. 

A  simplified  model  of  the  human  describing  function, 
termed  the  "cross-over"  model,  was  found  to  give  a  fairly  good 
representation  of  the  human  operator  under  certain  circum- 
stances.  It  consisted  principally  of  a  pure  operator  gain 
(K  )  and  an  effective  time  delay  (t  ).   This  x   term  is  shown 
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to  be  a  random  variable  consisting  of  various  operator  delays, 
lags  and  high  frequency  leads. 

Jex  et  al.  [Ref.  9]  proved  that  when  an  operator  is 
controlling  a  first-order  divergent  element  with  transfer 
function   Y  (s)  =  - — r   ,  the  maximum  value  of  X  under  which 

O  S—  A 

control  can  be  maintained  is  approximately  equal  to  the 
inverse  of  the  operator's  effective  time  delay.   This 
instability  level  is  termed  "critical"  (A  )  and  provides  a 
simple  and  direct  measure  of  an  operator's  tracking  ability. 

The  task  in  which  A  is  slowly  and  monotonically 
increased  during  a  run  until  control  is  lost  is  called  the 
"critical  task."   A  similar  task  in  which  A  is  maintained 
at  some  controllable  value  is  called  the  "sub-critical  task." 

Applications  of  this  critical  task  appear  limitless 
in  the  area  of  human  response  research.   It  is  currently 
employed  by  a  number  of  experimenters  as  an  indicator  of 
psychomotor  performance.   In  particular,  Jex  [Ref.  10]  has 
applied  this  task  to  a  two-axis  tracking  system  in  which  the 
primary  task  consisted  of  the  critical  task.   The  separate 
secondary  task  consisted  of  the  sub-critical  task.   He  demon- 
strated that  noticeable  decreases  in  the  value  of  A   achieved 
in  the  primary  task  were  obtained  with  only  slight  increases 
in  the  secondary  level  of  instability,  and  that  the  combined 
tasks  could  be  used  for  workload  research. 

B.   PSYCHOPHYSIOLOGICAL  MEASUREMENT 

"Psychophysiological"  is  a  combination  of  the  adjectives 
psychological  and  physiological  and  refers  to  the  inseparable 
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connection  of  the  mental  and  neuromuscular  responses  of  the 
human  operator.   Since  these  responses  are  considered  to  be 
directly  related  to  the  operator's  state  of  stress  (workload), 
they  have  been  measured  quite  extensively  as  both  monitors 
and  indicators  of  workload.   Analyses  of  such  psychophysio- 
logical responses  as. heart  and  respiration  rates,  galvanic 
skin  response,  electrocardiograms  and  electroencephalograms 
have  been  performed. 

Spyker  et  al.  [Ref.  4]  performed  a  complex  analysis  of 
sixty-four  such  responses  during  a  two-axis  tracking  task. 
Using  a  considerable  amount  of  interface  and  data  processing 
equipment  he  arrived  at  a  reliable  workload  index  consisting 
of  ten  of  these  responses. 

Kahneman  [Ref.  11]  recorded  and  compared  just  three 
responses,  including  the  somewhat  unusual  measurement  of 
changes  in  the  subject's  pupil  size,  during  a  paced  mental 
task.   He  observed  that  this  pupil  size  variation  provided 
the  most  consistent  results. 

The  technique  of  pupillometry ,  or  the  method  of  measuring 
changes  in  pupil  size,  has  primarily  remained  in  the  realm  of 
the  behavioral  scientist.   The  preliminary  work  of  Hess  and 
Polt  [Ref.  12]  initiated  a  surge  of  research  in  this  area 
which  resulted  in  the  acclamation  of  pupillometry  as  a  very 
effective  indicator  of  the  human  condition.   It  has  been 
shown  to  indicate  an  individual's  attitudes  and  preferences 
[Ref.  13],  prejudices  [Ref.  14],  sexual  arousal  [Ref.  15], 
even  latent  homosexual  tendencies  [Ref.  16].   Pupillary 


16 


variations  have  been  used  to  indicate  psychiatric  disorders 
[Ref.  17],  as  well  as  physical  ills  [Ref.  18].   Hess  [Ref.  19] 
provides  an  excellent  assessment  and  summary  of  the  more 
important  research  performed  in  these  areas. 

Cognitive  demand  experiments,  i.e.,  those  requiring  mental 
effort,  have  also  been  performed  and  related  pupil  changes  to 
levels  of  mental  activity.   Hope  [Ref.  20]  showed  that  in- 
creasing levels  of  difficulty  in  a  mental  arithmetic  problem 
resulted  in  proportionate  pupil  dilations  and  noted  the 
phenomena  of  pupillary  constriction  when  the  difficulty  level 
was  excessive,  causing  incorrect  replies.   He  referred  to 
this  as  a  mental  "overload." 

Payne  et  al.  [Ref.  21]  reported  that  changes  in  pupil 
diameter  were  a  sensitive  and  reliable  indicator  of  informa- 
tion processing  capacity  and  were  related  to  levels  of 
workload . 

Edwards  [Ref.  22]  showed  similar  correlation  with  infor- 
mation processing  and  reported  the  same  pupil  constriction 
for  mental  overloads  as  shown  by  Hope  [20]. 
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III.   EXPERIMENT 

A.   APPLICATION  OF  THEORY 
1.   Tracking  Tasks 

a.   Cross-Adaptive 

Two  variations  of  the  critical  task  were  used  as 
the  primary  and  secondary  tasks  in  a  cross-adaptive  system. 
The  sub-critical  task  served  as  the  controlled  element  of  the 
primary  task,  and  a  variable  sub-critical  (i.e.,  where  X 
varies  continuously  at  levels  below  X  )  was  used  as  the 
secondary  (loading)  task.   This  was  done  for  several  reasons: 

(1)  The  critical  task  is  simple,  easily 
mechanized  and  can  be  used  on  a  small  analog  computer. 

(2)  The  level  of  instability  (X)  is  directly 
related  to  the  level  of  difficulty  of  the  task  and  can  be 
used  both  to  set  the  level  of  difficulty  for  the  primary  task 
(the  basis  for  workload  comparison)  and  to  vary  the  level  in 
the  secondary  task  (thus  becoming  the  adaptive  variable). 

(3)  The  tasks  required  no  input.   The  random 
operator  remnant  is  sufficient  to  excite  and  maintain  the 
unstable  tasks. 


Although  the  term  "critical"  specifically  refers  to  the 
critical  instability  task  in  which  X  is  monotonically 
increased  until  reaching  X  ,  it  has.  been  common  to  refer  to 
the  general  (i.e.,  first-order  divergent)  compensatory 
tracking  task  from  which  x   is  obtained  as  the  critical  task. 
Unless  otherwise  noted,  the  author  has  used  the  term  in  this 
general  sense. 
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(4)  In  addition  to  its  function  as  the 

adaptive  variable,  the  secondary  instability  becomes  the 

indirect  measure  of  primary  task  workload. 

The  performance  measure  employed,  although  not 

as  sophisticated  as  desired,  was  the  deviation  of  the  absolute 

value  of  the  primary  task  error  (e  )  from  a  fixed  criterion 

(e~).   The  criterion  was  established  by  determining  the  rms 

error  of  the  principal  subject  during  tracking  runs  at  a 

median  level  of  primary  task  instability  ( x  )  and  was  set  at 

0.5  cm.  (of  displayed  error).   Thus  primary  task  performance 

(P  )  is  defined: 
P 

P   =   e~  -  le  I  (2) 

p      f    '  p ' 

The  adaptive  logic  used  was  similarly  simple. 

dXs 
The  time  rate  of  change  of  the  adaptive  variable  (  -rr—  )  was 

related  to  P   in  a  manner  similar  to  Kelley's  approach 

[Ref.  7,  p.  55^]  i:e., 


dx„ 

^  =   K  P^  (3) 

dt         p 

where  the  gain  (K)  is  set: 

K    =   0.4  rad./cm.-sec.  (4) 

To  prevent  excessive  instability  rates  and  to 
provide  the  operator  with  a  greater  opportunity  to  regain 
control,  the  rate  was  limited  in  the  following  manner: 


dX 
s 

dt 


=   0.14  rad./sec?,   P   >.  0.35    (5) 
max 
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dt 


=  -0.26  rad./sec?,  P   <_  -0.65    (6) 

max  P 


Thus  when  the  operator  exceeded  the  criterion, 
the  secondary  task  instability  rate  decreased  almost  twice  as 
fast  as  it  increased  when  he  tracked  within  the  criterion. 

A  further  limit  was  placed  on  the  instability  to 
prevent  the  secondary  task  from  ever  becoming  stable  which, 
in  this  particular  mechanization,  would  cause  control 
reversal.   Thus: 

xs|  .    =   0.16  rad./sec.  (7) 

'rain 

An  initial  value  of  instability  (X   )  was  provided 

o 
to  insure  that  the  operator  was  loaded  in  both  axes  at  the 

beginning  of  the  run.   This  relatively  low  initial  value  was: 

xs   =   1.0  rad./sec.  (8) 

A  diagram  of  the  cross-adaptive  system  is  shown 
in  Fig.  (2). 

b.   Single-Axis  Sub-Critical 

In  order  to  provide  a  common  ground  for  comparing 
the  two  different  methods  of  workload  measurement,  the  pri- 
mary task  alone  had  to  be  performed  for  pupillometric 
measurements.   By  disconnecting  both  the  secondary  task  and 
the  cross-adaptive  network,  the  single-axis  sub-critical  task 
was  generated. 
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c.   Single-Axis  Critical 

The  critical  task2  was  used  to  determine  each 
subject's  tracking  skill  and  level  of  proficiency  during  the 
experiment.   It  was  used  to  normalize  measured  instability 
levels  obtained  during  the  tracking  tasks  and  to  indicate  the 
level  of  total  control  capacity. 

Although  an  automatic,  rate-switched  mechanism 
(autopacer)  was  recommended  by  Jex  et  al.  [9],  this  was  not 
possible  in  this  experimental  set-up  and  the  value  of  lQ   had 
to  be  obtained  by  manually  increasing  the  level  of  instability 
on  the  task  from  zero  to  the  point  where  control  was  lost. 
2.   Pupil  Diameter 

The  results  obtained  by  Kahneman  [11]  and  by 
Anderson  [Ref.  23]  led  to  the  selection  of  pupil  diameter 
changes  as  the  psychophysiological  indicator  of  workload. 
The  authors  indicated  a  correlation  between  level  of  diffi- 
culty and  pupil  size  variation  and  utilized  "information 
processing  capacity"  in  a  manner  analogous  to  the  use  of 
"reserve  capacity"  in  this  study. 

B.   PROCEDURE 

Three  subjects3,  chosen  on  the  basis  of  motivation, 
availability  and  previous  tracking  or  flying  experience, 


2Now  used  in  its  specific  sense  (i.e.,  ^O- 


3Several  other  subjects  were  considered  to  provide  a 
larger  and  more  meaningful  data  base.   After  several  trial 
runs  they  had  to  be  rejected  due  to  eye  characteristics  (i.e., 
little  contrast  between  iris  and  pupil  or  partial  covering  ol 
the  pupil  by  the  eyelid)  which  caused  the  pupil  measuring 
equipment  to  give  erroneous  readings.   A  similar  difficulty 


?quipme 
was  reported  by  Burns  [24]. 
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performed  the  critical  tracking  tasks  described  previously. 
The  tasks  were  mechanized  on  a  small  analog  computer  accord- 
ing to  the  circuit  diagram  shown  in  Fig.  (1*0 .   Tracking 
errors  were  displayed  as  the  displacement  of  a  dot  from  the 
center  of  a  cathode  ray  tube  (CRT)  located  15.5  in.  (39. ^  cm.) 
in  front  of  the  subject's  eye. 

Control  was  affected  with  an  isometric  (force)  stick. 
The  controller  was  mounted  on  a  tablet-  arm  to  provide  forearm 
support  and  positioned  slightly  ahead  and  to  the  right  of  the 
subjects,  who  were  all  right-handed.   To  reduce  operator 
fatigue  and  to  allow  higher  response  frequencies,  the  sensi- 
tivity of  the  controller  was  set  at  a  fairly  high  level. 
The  ratio  of  display  deflection  to  control  force  was  2.84 
in. /lb.  (1.62  cm./N). 

Continuous  pupil  diameter  measurements  were  made  using 
a  Space  Sciences  Model  831  Television  Pupillometer .   This 
device  uses  a  closed-circuit  television  system  to  observe 
the  eye  and  a  signal  processing  unit  to  measure  and  display 
pupil  diameter.   The  eye  was  illuminated  by  a  small,  near- 
infrared,  low-intensity  light  source  which  was  mounted  to 
one  side  of  the  adjustable  camera.   The  subject's  head  was 
positioned  in  a  brace  consisting  of  a  chin  support  and  a 
forehead  restraint  (Fig.  3).   An  integral  two-channel  strip 
chart  recorder  in  the  processing  unit  recorded  pupil  diameter 
in  a  range  from  zero  to  ten  millimeters.   The  second  channel 
was  used  to  record  either  secondary  task  instability  (*s) 
during  the  cross-adaptive  task,  or  displayed  error  (e  ) 


22 


during  the  sub-critical  tracking  runs.   An  additional 
recorder  was  also  used  to  record  total  displayed  error 
(eT  =  |e  |  +  |e  |)  during  the  cross-adaptive  runs.   A  picture 
of  the  experimental  set-up  is  shown  in  Fig.  (4). 

Seated  before  the  display,  each  subject  positioned  his 
head  comfortably  in  the  restraint  while  focusing  on  the 
center  of  the  elevated  CRT.   The  display  was  raised  14.5 
degrees  above  eye  level  to  force  the  subject  to  look  up, 
thus  providing  better  illumination  of  the  eye  and  removing 
any  interference  between  the  eyelid  and  pupil. 

The  experiments  were  performed  in  an  acoustically  pro- 
tected chamber  with  constant,  dim  back  lighting.   The  chamber 
was  located  in  the  Human  Engineering  Laboratory  of  the  Naval 
Postgraduate  School. 

The  experiments  were  divided  into  three  phases,  each 

corresponding  to  the  different  tracking  tasks  to  be  performed 

During  the  initial  phase  the  subject  was  given  a  warm-up 

period  to  become  familiar  with  the  task  and  to  achieve  a 

relatively  stable  performance  level.   He  was  then  asked  to 

perform  the  cross-adaptive  critical  task,  and  reminded  that 

control  of  the  primary  task  was  paramount.   This  meant  that 

he  should  try  to  keep  the  vertical  position  of  the  dot 

(indicating  e  )  in  the  center  of  the  CRT  as  much  as  possible 

or  to  center  it  as  quickly  as  possible  when  displaced.   He 

was  requested  to  keep  the  horizontal  position  of  the  dot 

(e  )  at  least  within  five  cm.  of  the  center, 
s 
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A  series  of  runs,  each  lasting  from  90  to  150  seconds, 
were  then  performed  at  various  levels  of  primary  task  insta- 
bility.  Table  la.  indicates  the  number  of  runs  performed  by 
the  subjects  at  each  of  these  levels. 

Recordings  of  secondary  task  instability  and  total  error 
were  made  during  each  run.   Although  this  experimental  phase 
was  primarily  aimed  at  obtaining  values  of  A   as  a  workload 
indicator,  pupillary  reactions  were  recorded  to  determine  if 
any  correlation  existed  between  pupil  dilation  and  X  . 

In  the  second  experimental  phase,  subjects  performed  the 
critical  instability  task  in  the  vertical  axis.   Each  subject 
performed  the  task  five  times  with  run  lengths  averaging  20 
to  30  seconds.   Values  of  X   were  recorded  along  with  the 
subsidiary  measurement  of  pupil  diameter.   Figure  (5)  shows 
a  typical  time  history  of  the  critical  task  instability 
levels  and  pupillary  variations. 

The  final  phase  of  the  experiment  consisted  of  having  the 
subjects  perform  a  series  of  sub-critical  single-axis  tracking 
tasks  at  the  same  instability  levels  (A  )  as  used  in  the 
cross-adaptive  tasks.   Measurements  of  pupil  diameter  and  of 
displayed  error  (e  )  were  recorded.   The  number  of  runs  per 
subject  at  each  level  of  primary  instability  is  shown  in 
Table  lb. 

C.   DATA  ANALYSIS 

To  reduce  the  effects  of  inter-subject  variation,  the 
recorded  data  was  normalized.   Instability  levels  were 
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represented  as  a  percentage  of  the  subject's  mean  critical 
value  of  instability  (T  ),  i.e., 

XS  s   =  -4^"  x   100  (9) 

Ac 

A  procedure  recommended  by  Pratt  [Ref.  25]  and  one  commonly 
used  in  pupillometric  measurements  was  to  express  the  pupil 
diameter  (D)  as  a  percentage  change  from  the  average  unloaded 
initial  diameter  (D~~ ) ,  i.e., 

D  -  D~~ 
D*   = x   100  (10) 

Do 

The  determination  and  averaging  of  this  initial  diameter, 
however,  was  far  from  simple.   Figure  (9)  shows  the  wide  and 
random  variation  of  one  subject's  pupil  diameter  during  two 
periods  of  rest  following  consecutive  tracking  runs.   These 
effects  are  caused  both  by  a  small  amount  of  random  noise 
introduced  in  the  optical  neuromuscular  system  (akin  to 
McRuer's  "remnant"  [8])  as  described  by  Stanten  and  Stark 
[Ref.  26]  and  by  operator  cognitive  processes  (i.e.,  day- 
dreaming).  The  latter  effect  is  presumed  to  be  predominant 
and  is  eliminated  when  the  operator  is  loaded  and  forced  to 
pay  full  attention  to  the  tracking  task. 

The  figure  also  indicates  that  there  is  a  significant 
difference  between  the  mean  values.   Thus  a  determination  of 
D   is  required  for  each  separate  run. 

In  order  to  preclude  an  inordinate  amount  of  tracking  and 
data  reduction  time,  this  type  analysis  (i.e.,  numerically 
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averaging  pupil  diameter  every  three  seconds  during  an  initial 
90  second  period)  was  not  performed.   Instead,  a  15-30  second 
rest  period  before  each  run  was  used  as  the  baseline  and  the 
pupil  recordings  were  visually  averaged  to  obtain  D~~. 

Analysis  of  pupil  measurements  recorded  during  both  the 
cross-adaptive  and  sub-critical  tracking  tasks  required 
similar  averaging  techniques.   Although  not  elegant  from  an 
engineering  viewpoint,  this  method  provided  adequate  and 
representative  mean  data.   It  may  have  been  partially  respon- 
sible, however,  for  introducing  scatter  in  the  data. 

During  the  analysis  of  the  instability  data,  a  consist- 
ently recurring  phenomena  was  observed.   Peak  values  of 
secondary  instability  (X   ),  which  indicated  the  highest 
level  of  instability  achieved  before  the  primary  performance 
rapidly  deteriorated,  were  noted.   The  pupil  diameter  record- 
ing also  showed  a  similar  peak  (D   )  within  a  few  seconds. 
These  values  were  measured  and  recorded.   Figure  (7)  indicates 
this  phenomena  quite  clearly. 

Values  of  asymptotic  (steady  state)  secondary  instability 

(X   )  were  also  visually  averaged  over  the  observed  time 
s  s 

intervals  (At  )  during  which  they  were  maintained.   Pupil 

diameter  measurements  (D   )  were  also  averaged  during  this 

s  s 

interval,  and  the  values  for  each  were  recorded. 

Figures  (6)  to  (8)  show  typical  time  histories  of  the 
variables  for  the  three  tracking  tasks. 

All  data  was  normalized  and  averaged  for  each  subject 

at  each  level  of  x*.   Except  for  steady  state  values,  the 

P 
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mean  and  standard  deviation  was  calculated  using  standard 
statistical  formulae,  i.e., 


and, 


Where, 


x  = 


N 

Z 

i  =  l 


a  = 


N 


N    ?       N 

Z   x   -  (   Z   x  )  /N 


1/2 


i=l 


i  =  l 


N  -  1 


x  ■  A  ,  X*  ,  D*   D* 

c'   sm'    '   sm 


N  =  number  of  runs 


(11) 


(12) 


i.e. 


The  steady  state  values  were  weighted  by  the  percentage 

T 
At 


ratio  of  the  time  interval  (At  )  to  the  total  run  time  (trp), 


w  = 


t 


x  100 


(13) 


T 


Weighted  averages  and  standard  deviations  were  computed 
using  the  relations: 


N 

yw  '   N 

Z   w. 
i  =  l   X 


(14) 


and, 
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w 


N 
1=1  i  * 


N 
(  .J 

i=l 


2 
wiyi  ^  ^N 


N  -  1 


1/2 


(15) 


Where , 


ss >   ss 


The  averaged,  normalized  data  obtained  in  this  manner 
are  shown  in  Tables  II  to  IV. 
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IV.   RESULTS 

The  principal  results  of  this  experiment  are  presented 
in  Figs.  (10)  and  (11).   Using  the  cross-adaptive  critical 
tracking  task  as  an  indicator  of  workload,  Fig.  (10)  shows 
the  overall  decrease  in  the  steady  state  values  of  normalized 
secondary  instability  with  increasing  normalized  primary 
instability.   The  demonstrated  relationship  between  secondary 
instability  and  primary  workload  thus  validates  the  cross- 
adaptive  technique.   Simply  stated,  this  shows  that  as  the 
primary  task  becomes  more  difficult,  the  level  of  diffi- 
culty maintained  in  the  secondary  task  becomes  less,  indicating 
the  previously  predicted  increase  in  primary  task  workload. 
Thus,  the  task  difficulty  is  reflected  by  its  workoad  and 
vice  versa. 

The  results  of  using  changes  in  pupil  diameter  as  a 
measure  of  workload  are  shown  in  Fig.  (11)  .   The  proportional 
relationship  between  level  of  difficulty  and  increase  in 
pupil  diameter  is  evident  and  thus  reflects  the  increasing 
workload.   An  apparent  increase  in  the  rate  of  change  of 
pupil  size  is  noted  at  approximately  50  percent  of  the  total 


Data  are  presented  in  this  figure  for  only  two  subjects. 
The  third  subject  became  rapidly  fatigued  during  this  experi- 
mental phase.   This  fatigue  caused  his  eyelids  to  droop 
slightly  which  in  turn  interfered  with  the  correct  measure- 
ment of  his  pupil  diameter. 
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operator  control  capacity  and  indicates,  perhaps,  a  mental 
"shifting  of  gears"  at  the  midpoint  of  this  capacity. 

Due  to  the  large  variance  in  the  data,  no  regression 
analyses  were  performed.   The  mean  trends,  however,  are 
clearly  evident  and  support  the  theoretical  hypothesis. 

Additional  results  obtained  from  the  experiment  are  shown 
in  Figs.  (12)  and  (13).   The  variance  of  peak  values  of 
normalized  primary  instability  is  shown  in  Pig.  (12).   The 
same  trend  as  shown  previously  in  Fig.  (10)  is  evident.   This 
parameter  could  thus  be  used  as  an  additional  workload  index. 

Fig.  (13)  shows  the  results  of  the  critical  task  measure- 
ments.  It  can  be  seen  that  two  subjects  dilated  at  fairly 
constant  rates  until  reaching  approximately  90  percent  of 
critical,  at  which  time  the  dilation  rate  greatly  increased 
until  reaching  critical.   The  unexpected  phenomenon  of  initial 
pupillary  constriction  with  increasing  instability  is  ob- 
served in  one  subject,  and  can  be  explained  in  light  of  his 
admitted  tracking  strategy. 

This  subject  had  previously  performed  a  large  number  of 
critical  instability  tracking  tasks  and  had  developed  a 
strategy  which  enabled  him  to  achieve  consistently  high 
values  of  X  .   By  consciously  relaxing  during  the  early 
phases  of  the  run  when  tracking  is  relatively  simple,  he  would 
store  control  energy  for  the  highly  unstable  regions 
(essentially  switching  from  a  low  to  a  high  gain).   The 
pupillometer  recorded  this  reduction  in  stress  (workload)  by 
either  an  accurate  measurement  of  actual  pupil  constriction 
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or  by  the  inaccurate  measure  of  the  pupil  partially  blocked 
by  a  lowered  (restful)  eyelid.   Dilation  occurred  when  the 
instability  loaded  the  subject  beyond  his  capacity  to  relax 
and  is  shown  to  have  a  similarly  high  rate  near  critical. 

One  additonal  result  was  obtained  with  the  critical  task 
As  indicated  in  Fig.  (5),  rapid  pupil  constriction  following 
loss  of  control  is  evident.   This  is  a  manifestation  of  the 
"overload"  condition  of  Hope  [20]  and  Edwards  [22]. 

The  additional  measurements  of  changes  in  pupil  diameter 
during  the  cross-adaptive  task  at  corresponding  levels  of 
steady  state  and  peak  secondary  instability,  provided  no 
discernable  information.   It  is  assumed  that  the  complex 
interaction  of  primary  and  secondary  tracking  tasks  as  seen 
through  the  eye,  made  analysis  of  pupillary  responses 
fruitless . 

The  results  obtained  from  the  cross-adaptive  critical 
tracking  task  have  been  compared  to  those  of  Jex  et  al. 
[Ref.  27].   Although  not  specifically  employed  to  indicate 
operator  workload,  their  more  sophisticated  cross-adaptive 
task,  coupled  with  measurements  of  operator  describing 
functions,  provided  three  results  similar  to  those  of  this 
experiment : 

1.  Three  primary  task  controlled  elements  of  in- 
creasing difficulty  resulted'  in  decreasing  levels  of  cross- 
coupled  instability. 

2.  Small  increases  in  effective  time  delay  and 
remnant  were  measured. 
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3.   Operator  behavior  (tracking  strategy)  in  the 
primary  task  as  determined  by  describing  function  measure- 
ments was  not  changed  by  secondary  task  loading,  indicating 
that  he  had  not  changed  his  primary  task  performance  criterion 
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V.   CONCLUSIONS  AND  RECOMMENDATIONS 

The  following  conclusions  concerning  this  research  are 
drawn: 

1.  This  thesis  has  been  successful  in  mechanizing 
two  techniques  for  determining  the  level  of  workload  of  an 
operator  controlling  a  manual  control  system. 

2.  The  techniques  of  using  the  cross-adaptive  criti- 
cal tracking  task  and  of  measuring  changes  in  pupil  size 
provide  valid  indices  of  operator  workload. 

3.  The  cross-adaptive  task  can  be  mechanized  on  a 
small  computer  using  simple  performance  measurements  and  logic 
which  will  yield  results  comparable  to  those  obtained  from 
larger }  more  complex  systems. 

4.  An  additional  workload  index,  the  peak  value  of 
secondary  task  instability,  is  available  and  can  be  used  to 
confirm  results  obtained  from  the  two  principal  indices. 

5.  The  measurement  of  critical  instability  levels 
provides  a  good  measure  of  an  operator's  total  control 
capacity  and  enables  the  workload  index  to  be  expressed 
concretely. 

6.  Pupil  measurements  are  highly  inter-  and  intra- 
subject  variant.   Additional  variance  may  have  been  introduced 
by  a  relatively  crude  method  of  data  reduction. 

7.  The  simultaneous  measurement  of  cross-adaptive 
instability  and  changes  in  pupil  diameter  provides  no 
meaningful  data. 
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The  following  recommendations  are  offered: 

1.  The  cross-adaptive  tracking  task  should  be  coupled 
with  the  recently  developed  hybrid  describing  function  ana- 
lyzer [Ref.  28]  as  a  more  effective  measure  of  workload. 

2.  The  use  of  a  smoothing  filter  and  of  similar 
performance  measurement  techniques  as  presented  in  Ref.  27 
should  be  incorporated  in  the  cross-adaptive  mechanization. 

3.  A  more  realistic  and  less  fatiguing  display 
format  should  be  used  (e.g.,  a  simulated  artificial  horizon). 

4.  A  method  of  interfacing  pupillometric  output  with 
an  off-line  data  processor  including  programs  for  data 
averaging  should  be  found.   This  would  make  pupillometry  more 
appealing  as  a  workload  measure  for  tracking  tasks. 
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APPENDIX  A 
TABLES 

TABLE  I 
Number  of  Experimental  Runs  per  Subject 


a. 
Cross-Adaptive  Tracking  Task 

Primary  Task  Instability  (Ap) 


Subj  ect 

1.0 

1.5 

2.0 

2.5 

3.0 

3.5 

4.0 

RAH 

- 

- 

6 

11 

15 

8 

5 

HGC 

- 

- 

4 

3 

3 

- 

- 

CMB 

5 

5 

5 

_ 

5 

_ 

— 

b. 
Sub-Critical  Tracking  Task 

Primary  Task  Instability  (xp) 
Subject    1.0    1.5    2.0    2.5    3.0    3.5    4.0 

HGC  4     3     3     -     3 

CMB      3     3     3     3     3-3 
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TABLE  II 

Normalized  Instability  Levels  and  Pupil  Diameters 
for  Sub-Critical  Tracking  Task 


Normalized  Primary         Normalized 
Subject      Instability  Up*)      Pupil  Diameter  (D*) 


HGC 


CMB 


(mean 

/  std .  dev. ) 

32.34 

7-85 

(2.60) 

40.43 

12.60 

(0.70) 

48.51 

12.38 

(1.09) 

64.70 

18.12 

(5.42) 

21.93 

3-53 

(9.50) 

32.89 

5.52 

(2.17) 

43.86 

10.94 

(2.58) 

65.79 

35.79 

(22.86) 

87.70 

50.03 

(5.68) 
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TABLE  III 

Normalized  Instability  Levels 
for  Cross-Adaptive  Critical  Tracking  Task 


Normalized  Steady  Normalized  Peak 

Normalized  Primary   State  Secondary       Secondary 

Subj  .    Instability  (X  *)   Instability  (X*   )  Instability  (X*  ) 

p  s  s  sm 


RAH 

31.64 

39.56 

47.^7 

55.38 

63.29 

HGC 

32.32 

40.43 

48.51 

CMB 

21.93 

32.89 

43.86 

65.79 

(mean  /  standard  deviation) 

65.06  (9-93)  74.19  (10.15) 

65.03  (4.59)  76.58  (6.41) 

62.75  (11.06)  75.68  (12.97) 

43.16  (13.47)  62.18  (19.45) 

30.66  (15.44)  35.24  (19.46) 

58.02  (11.23)  55.63  (10.58) 

68.20  (3-00)  56.89  (6.58) 

42.65  (8.22)  55.63  (10.58) 

79.32  (13.58)  90.34  (9-34) 

62.79  (9.39)  75.29  (9.80) 

27.43  (21.10)  42.32  (22.9) 

7.04   (3. 90)  29.0  (14.17) 
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TABLE  IV 

Normalized  Pupil  Diameters 
for  Cross-Adaptive  Critical  Tracking  Task 


Subj  . 


Normalized  Steady   Normalized  Peak 
Normalized  Primary     State  Pupil  Pupil 

Instability  (A  *)    Diameter  (D*  )    Diameter  (D*  ) 


RAH 

31.64 

4.63 

(5.13) 

35.95 

(24.99) 

39.56 

2.87 

(4.19) 

18.87 

(12.75) 

47.^7 

0.82 

(5.53) 

14.35 

(8.86) 

55.38 

-5.35 

(7.98) 

2.69 

(6.78) 

63.29 

-2.56 

(8.70) 

12.36 

(5.36) 

HGC 

32.34 

18.66 

(4.34) 

18.23 

(4.84) 

40.43 

16.65 

(4.60) 

6.86 

(1.51) 

48.51 

3.03 

(7.77) 

18.04 

(6.37) 

CMB 

21.93 

15.06 

(13.02) 

36.45 

(15.73) 

32.89 

-4.12 

(20.82) 

19.50 

(15.56) 

43.86 

-6.35 

(12.9D 

6.86 

(8.81) 

65.79 

2.58 

(17.47) 

14.10 

(15.56) 
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Close-Up  of  Subject  Tracking 
(indicating  TV  Pupillometer ) 

Figure  (3) 


Experimental  Set-Up 
Figure  (4) 
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Figure  (10) 
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